A Multi-Method Approach for Discriminating Between Similar Facial Expressions, Including Expression Intensity Estimation

Facial expression provides sensitive cues about emotion and plays a major role in interpersonal and humancomputer interaction. Most facial expression recognition systems have focused on only six basic emotions and their concomitant prototypic expressions posed by a small set of subjects. In reality, humans are capable of producing thousands of expressions that vary in complexity, intensity, and meaning. To represent the full range of facial expression, we developed a computer vision system that automatically recognizes individual action units (AUs) or AU combinations using Hidden Markov Models and estimates expression intensity. Three modules are used to extract facial expression information: (1) facial feature point tracking, (2) dense flow tracking with principal component analysis (PCA), and (3) high gradient component detection (i.e. furrow detection). The average recognition rate of upper and lower face expressions is 85% and 88%, respectively, using feature point tracking, 93% (upper face) using dense flow tracking with PCA, and 85% and 81%, upper and lower face respectively, using high gradient component detection.